1. Field of the Invention
The present invention relates generally to a method for acquiring multicolor images by forming a Red, Green, and Blue (RGB) channel or RGB color on each pixel, based on image data entered by passing through a Bayer color filter array, and more particularly, to an edge-adaptive interpolation and noise filtering method performed for the image data.
2. Description of the Related Art
Image sensors used in portable terminals with camera modules include Complementary Metal-Oxide Semiconductor (CMOS) image sensors and Charge-Coupled Device (CCD) image sensors. A CMOS image sensor is used in most cellular phone cameras and also in low-cost digital cameras because it has a high degree of integration and is easily mass-produced. Although the CMOS image sensor has advantages of low cost and low power usage, it is weak in noise. Most of this noise is processed by software to correct pixel values of images output from the image sensor.
A fundamental technique of a picture quality improvement algorithm is to detect an edge within an image and to form an RGB value using neighborhood pixel values, without artifacts for each pixel, in consideration of characteristics of the edge.
Some of the available interpolation and noise filtering methods include interpolation methods considering edge direction and interpolation methods using a noise elimination structure.
For interpolation methods considering edge direction, an edge direction is estimated and interpolation is performed along the estimated edge direction. However, there are various edge analysis methods and interpolation methods using neighborhood pixel values.
For example, edge-adaptive interpolation methods supporting two directions, such as U.S. Pat. No. 5,373,322, which issued to Laroche et al., and is entitled “Apparatus and Method for Adaptively Interpolating a Full Color Image Utilizing Chrominance Gradients”, and U.S. Pat. No. 6,507,364, which issued to Bishay et al., and is entitled “Edge-dependent Interpolation Method for Color Reconstruction in Image Processing Devices”, disclose interpolation methods for obtaining a gradient by dividing an edge direction for a 3×3 window into horizontal and vertical directions and performing, interpolation using the gradient. Additionally, U.S. Pat. No. 6,882,563, which issued to Asao, and is entitled “Magnetic Memory Device and Method for Manufacturing the Same”, discloses a method for eliminating a color fringe by adding a correction step, after performing interpolation similarly to the above-disclosed interpolation methods.
Edge-adaptive interpolation methods supporting four directions, such as U.S. Pat. No. 6,404,918, which issued to Hel-or et al., and is entitled “Image Demosaicing Method Utilizing Directional Smoothing”, discloses a method for extracting directions using a steerable filter and iterating interpolation in the extracted direction. Additionally, U.S. Pat. No. 6,832,009, which issued to Shezaf et al., and is entitled “Method and Apparatus for Improved Image Interpolation”, discloses an interpolation method for detecting a four-direction edge using a 3×3 Sobel operation.
Further, U.S. Pat. No. 6,707,937, which issued to Sobel et al., and is entitled “Interpolation of Edge Portions of a Digital Image”, discloses an interpolation method using 3×3 edge detection of short scale and long scale filtering. U.S. Pat. No. 7,133,553, which issued to Embler, and is entitled “Correlation-based Color Mosaic Interpolation Adjustment Using Luminance Gradients”, discloses an interpolation method performed by measuring an edge direction using a 3×3 preset directional filter. U.S. Pat. No. 7,376,288, which issued to Huang et al., and is entitled “Edge Adaptive Demosaic System and Method”, discloses a method for performing interpolation in a gradient direction within a threshold after obtaining a gradient value of four edge directions within a 5×5 window.
For interpolation methods using a noise elimination structure, noise may occur due to a Gr/Gb difference or an interpolation direction measurement error. Because it is unstable to accurately measure an edge direction or intensity at a Bayer level, noise is generally eliminated after performing interpolation and then measuring the accurate edge direction and intensity.
For example, U.S. Pat. No. 6,795,586, which issued to Gindele et al., and is entitled “Noise Cleaning and Interpolating Sparsely Populated Color Digital Image”, discloses a structure in which color interpolation and noise elimination are processed in one block. U.S. Pat. No. 6,816,194, which issued to Zhang et al., and is entitled “Systems and Methods with Error Resilience in Enhancement Layer Bitstream of Scalable Video Coding”, discloses an interpolation method using a boundary of an edge by application of a bilateral filter during a demosaicing process.
Similarly, U.S. Pat. No. 7,256,828, which issued to Nilsson et al., and is entitled “Weighted Gradient Based and Color Corrected Interpolation”, discloses a structure using the intensity of an edge as an interpolation weight. U.S. Pat. No. 6,970,597, which issued to Olding et al., and is entitled “Method of Defining Coefficients for Use in Interpolating Pixel Values”, discloses a structure for determining a coefficient of a kernel by raising window support during interpolation.
Other interpolation methods include, for example, U.S. Pat. No. 7,292,725, which issued to Chen et al., and is entitled “Demosaicking Method and Apparatus for Color Filter Array Interpolation in Digital Image Acquisition Systems”. U.S. Pat. No. 7,292,725 discloses a structure in which a neighborhood pixel value and an interpolation value are tabulated and the interpolation value is retrieved when a specific template is entered.
Additionally, U.S. Pat. No. 6,130,960, which issued to Acharya, and is entitled “Block-matching Algorithm for Color Interpolation” and U.S. Pat. No. 6,933,971, which is issued to Bezryadin, disclose interpolation methods using a weighted average, after analyzing the intensity of a neighborhood pixel.
U.S. Pat. No. 7,053,944, which issued to Acharya, and is entitled, “Method of using hue to interpolate color pixel signals”, discloses an interpolation method using a characteristic showing uniform hue value in the same color region.
However, the conventional interpolation and filtering methods above have a number of problems.
First, an edge direction is not accurate detected during edge detection in a Bayer pattern. Because it is important to naturally restore images while preserving a boundary of an edge, it is also important to accurately analyze an edge direction within a given window or block.
Second, inaccurate edge detection may generate noise during interpolation and may damage a detail/texture part during noise elimination. The inaccurate edge detection also affects detail extraction for edge enhancement.